Image-processing apparatus, image-processing system, image-processing method, and storage medium

ABSTRACT

According to one embodiment, an image-processing apparatus includes an output unit, a tracker, a storage controller, a setter, and an output controller. The tracker tracks a target object in one or more time-series images. The storage controller stores tracking information including a movement trajectory of the object in a predetermined storage. The setter sets a detection region for detecting passing through of the object based on the movement trajectory of the object, on the image, in accordance with a user&#39;s operation. An evaluator evaluates a setting mode for the detection region based on the tracking information stored in the predetermined storage and the detection region set by the setter. The output controller outputs the tracking information stored in predetermined storage, from the output unit, in an output mode based on an evaluation result of the evaluator.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2017-215983, filed Nov. 8, 2017, theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an image-processingapparatus, an image-processing system, an image-processing method, and astorage medium.

BACKGROUND

Hitherto, a technique has become known in which cameras are installed inthe vicinity of the entrance port of a building, and persons are trackedon images captured by the cameras, to detect that the tracked personsenter or leave rooms. In such a technique, it is determined whether aperson who is a target for tracking enters a room or leaves a room, forexample, in accordance with whether the person passes through adetection line. However, in the related art, in a case where thedetection line is set at a position where it is difficult to determinethe presence or absence of passing through, or the detection line is setat a position where it is determined that a person who does not enter orleave a room has passed through, there is a problem of a decrease in theaccuracy of detection of a person who enters or leaves a room.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing an example of a configuration ofan image-processing system of a first embodiment.

FIG. 2 is a diagram showing an example of a configuration of animage-processing apparatus of the first embodiment.

FIG. 3 is a flow diagram showing a flow of a series of processesperformed by a controller of the first embodiment.

FIG. 4 is a diagram showing a method of determining the presence orabsence of intersection between a detection line and tracklets.

FIG. 5 is a diagram showing a method of determining the presence orabsence of intersection between the detection line and tracklets.

FIG. 6 is a diagram showing a method of determining the presence orabsence of intersection between the detection line and tracklets.

FIG. 7 is a diagram showing a method of displaying tracklets.

FIG. 8 is a diagram showing a method of displaying tracklets.

FIG. 9 is a diagram showing a method of displaying tracklets.

FIG. 10 is a diagram showing a method of displaying tracklets.

FIG. 11 is a flow diagram showing a flow of a series of processesperformed by a controller of a second embodiment.

FIG. 12 is a diagram showing an example of a selection screen fortracklets.

FIG. 13 is a diagram showing an example of a setting screen for thedetection line.

FIG. 14 is a diagram showing an example of a screen on which thedetection line is set.

FIG. 15 is a diagram showing an example of tracklets displayed in adisplay mode based on each evaluation result.

FIG. 16 is a diagram showing an evaluation method in a case where atracklet and the detection line intersect each other.

FIG. 17 is a diagram showing an evaluation method in a case where thetracklet and the detection line intersect each other.

FIG. 18 is a diagram showing an evaluation method in a case where thetracklet and the detection line intersect each other.

FIG. 19 is a diagram showing an example of a setting mode for thedetection line.

FIG. 20 is a diagram showing an example of a setting mode for thedetection line.

FIG. 21 is a diagram showing an example of a setting mode for thedetection line.

FIG. 22 is a diagram showing an example of a setting mode for thedetection line.

FIG. 23 is a diagram showing another example of a setting mode for thedetection line.

FIG. 24 is a diagram showing another example of a setting mode for thedetection line.

FIG. 25 is a diagram showing a method of setting candidates for thedetection line.

FIG. 26 is a diagram showing an example of a screen on which preferencesfor a setting mode for the detection line are set.

FIG. 27 is a diagram showing a method of reflecting a setting mode for adetection line in a certain area under surveillance in a setting modefor a detection line in another area under surveillance.

FIG. 28 is a diagram showing an example of a hardware configuration ofthe image-processing apparatus of an embodiment.

DETAILED DESCRIPTION

According to one embodiment, an image-processing apparatus includes anoutput unit, a tracker (tracking unit), a storage controller (storagecontrol unit), a setter (setting unit), and an output controller (outputcontrol unit). The output unit outputs information. The tracker tracksan object which is a target in one or more time-series images. Thestorage controller stores tracking information, including movementtrajectories of one or more of the objects tracked by the tracker, in apredetermined storage. The setter sets a detection region for detectingpassing through of the object on the basis of the movement trajectory ofthe object, on the image, in accordance with a user's operation. Anevaluator evaluates a setting mode for the detection region on the basisof the tracking information stored in the predetermined storage and thedetection region which is set by the setter. The output controlleroutputs the tracking information stored in the predetermined storage,from the output unit, in an output mode based on an evaluation result ofthe evaluator.

Hereinafter, an image-processing apparatus, an image-processing system,an image-processing method, and a storage medium of an embodiment willbe described with reference to the accompanying drawings.

First Embodiment

FIG. 1 is a schematic diagram showing an example of a configuration ofan image-processing system 1 of a first embodiment. The image-processingsystem 1 of the first embodiment includes, for example, one or morecameras 10 and an image-processing apparatus 100. These apparatuses maybe connected to each other through, for example, a local area network(LAN) or the like. Each camera 10 is installed in the vicinity of, forexample, entrance ports located inside various buildings such ascommercial facilities or public facilities. More specifically, eachcamera 10 is installed on ceilings or the like in the vicinity of theentrance ports, and repeatedly captures an image of the surroundings atpredetermined time intervals from a viewpoint of looking down from itsinstallation position. The predetermined time interval is a timeinterval for a frame rate of, for example, 30 frames per second (FPS),60 FPS, 120 FPS, or the like. Each camera 10 generates a set oftime-series images, that is, a moving image by repeatedly capturing animage of the surroundings. This moving image may include, for example, acolorscale moving image, a grayscale moving image, or the like. Inaddition, for example, in a case where the camera 10 is provided with adistance sensor that measures a distance in a depth direction usinginfrared rays or the like, a moving image which is generated by thecamera 10 may include a distance moving image (depth moving image).

The image-processing apparatus 100 is installed, for example,individually in each building, and performs image processing on a movingimage generated by each of one or more cameras 10 installed in the samebuilding, to thereby detect persons who go in (enter a room) or personswho go out (leave a room) at each entrance port. For example, theimage-processing apparatus 100 sets a detection window W_(D) fordetecting a person who is a target for detection with respect to eachframe constituting a moving image, and scans this detection window W_(D)in the two-dimensional direction of a frame, to thereby detect at whichposition in a frame region the person is present. The image-processingapparatus 100 derives a trajectory obtained by linking detection windowsW_(D) in which a person is detected in each frame in a time direction,as a person's movement trajectory (hereinafter, referred to as atracklet TL), and determines whether a person enters an entrance port orleaves an entrance port on the basis of this tracklet TL and a detectionline LN_(D).

The image-processing apparatus 100 may perform such processes, forexample, on-line and in real time, and may store a moving image in astorage device followed by processing the moving image off-line.Meanwhile, the image-processing apparatus 100 may be installed inanother building without being limited to being installed in the samebuilding as a building in which the camera 10 is installed. In thiscase, the camera 10 and the image-processing apparatus 100 may beconnected to each other through a wide area network (WAN) or the like.In addition, some functions of the image-processing apparatus 100 may berealized by another computer (such as, for example, a cloud server)connected to a wide area network (WAN).

FIG. 2 is a diagram showing an example of a configuration of theimage-processing apparatus 100 of the first embodiment. Theimage-processing apparatus 100 includes, for example, a communicator(communication unit) 102, a display 104, an operator (operating unit)106, a controller 110, and a storage 130. The display 104 is an exampleof an “output unit”.

The communicator 102 includes a hardware interface such as a networkinterface card (NIC) capable of being connected to a network such as aLAN or a WAN. For example, the communicator 102 communicates with thecamera 10 through a network, and acquires a moving image from the camera10 of its communication partner.

The display 104 includes a display device such as, for example, a liquidcrystal display (LCD) or an organic electroluminescence (EL) display.The display 104 displays an image under control performed by thecontroller 110.

The operator 106 includes a user interface such as, for example, abutton, a keyboard or a mouse. The operator 106 accepts a user'soperation, and outputs information according to the accepted operationto the controller 110. Meanwhile, the operator 106 may be a touch panelconfigured integrally with the display 104.

The controller 110 includes, for example, an object-tracker(object-tracking unit) 112, a storage controller 114, an outputcontroller 116, a detection region-setter (detection region-settingunit) 118, and an evaluation and analyzer 120. These components of thecontroller 110 are realized by a processor such as a central processingunit (CPU) or a graphics-processing unit (GPU) executing a program(software) stored in the storage 130. In addition, some or all of thecomponents of the controller 110 may be realized by hardware (circuitry)such as a large-scale integration (LSI), an application-specificintegrated circuit (ASIC), a field-programmable gate array (FPGA), or agraphics-processing unit (GPU), and may be realized by cooperationbetween software and hardware. In addition, the above program may bestored in the storage 130 in advance, and may be stored on a detachablestorage medium such as a DVD or a CD-ROM and be installed in the storage130 from the storage medium by the storage medium being mounted in thedrive device of the image-processing apparatus 100.

The storage 130 is realized by, for example, a hard disk drive (HDD), aflash memory, an electrically erasable programmable read-only memory(EEPROM), a read-only memory (ROM), a random-access memory (RAM), or thelike. In the storage 130, moving images acquired from the camera 10,various processing results, and the like are stored in addition to aprogram referred to by a processor.

[Process Flow]

Hereinafter, the processing details of each component of the controller110 of the first embodiment will be described with reference to a flowdiagram. FIG. 3 is a flow diagram showing a flow of a series ofprocesses performed by the controller 110 of the first embodiment.Processing of the present flow diagram may be repeated, for example,whenever a moving image is acquired from the camera 10.

First, the object-tracker 112 tracks a target object in a moving imageacquired from the camera 10 (step S100). The target object may be, forexample, the upper half of a human body, or a part of a face, a head orthe like, and may be the whole body. For example, the object-tracker 112sets a detection window W_(D) in each frame constituting a moving image,and uses template matching for comparing an image region overlappingthis detection window W_(D) with a template image prepared in advance,to determine whether the image region overlapping the detection windowW_(D) is an image region reflecting a target object.

For example, the object-tracker 112 may perform template matching on thebasis of parameters such as a hue, a luminance value, or a luminancegradient. For example, in a case where a parameter of a hue is used intemplate matching, the object-tracker 112 may determine that the imageregion overlapping the detection window W_(D) is an image regionreflecting a target object in a case where a difference between the hueof the image region overlapping of the detection window W_(D) and thehue of the template image is equal to or less than a fixed value, anddetermines that the image region overlapping the detection window W_(D)is a background image region in a case where the difference exceeds thefixed value. In addition, for example, in a case where a parameter of aluminance value or a luminance gradient is used in template matching,the object-tracker 112 may determine that the image region overlappingthe detection window W_(D) is an image region reflecting a target objectin a case where a difference between the luminance value or theluminance gradient of the image region overlapping the detection windowW_(D) and the luminance value or the luminance gradient of the templateimage is equal to or less than a fixed value, and determines that theimage region overlapping the detection window W_(D) is a backgroundimage region in a case where the difference exceeds the fixed value.

The object-tracker 112 changes the position of the detection windowW_(D) after determining whether a region in which the detection windowW_(D) is set is an image region including a target object, anddetermines whether the region to which the position has been changed isan image region including a target object. The object-tracker 112repeats such processes to thereby detect a target object from each framewhile scanning the detection window W_(D) in the two-dimensionaldirection of a frame.

The object-tracker 112 compares the positions of target objects detectedin each frame between a plurality of frames consecutive in a time-seriesmanner, determines that these target objects are the same object in acase where a distance (distance on an image plane) between the positionswhich are targets for comparison is less than a predetermined distance,and determines that these target objects are different from each otherobject in a case where a distance between the positions of the targetobjects which are targets for comparison is equal to or greater than thepredetermined distance. The predetermined distance may, for example, bemade shorter as the frame rate of a moving image becomes higher, and maybe made longer as the frame rate becomes lower.

In addition, in a case where the shapes of target objects which aretargets for comparison are more than a certain degree similar to eachother, the object-tracker 112 may determine that these target objectsare the same object. For example, in a case where a difference betweenhues, luminance values, luminance gradients or the like is equal to orless than a fixed value, the object-tracker 112 may determine that theshapes of the target objects are similar to each other.

In addition, in a case where the position of a target object is detectedfrom each of a plurality of frames of a moving image acquired for acertain period in the past, the object-tracker 112 may obtain an opticalflow on the basis of the detected positions of the target object,predict the position of a target object at a certain time in the futureon the basis of the optical flow, compare the predicted position withthe position of a target object detected in reality at that time, anddetermine whether these target objects are the same object.

The object-tracker 112 links, between frames, representative points suchas the centroids of detection windows W_(D) set with respect to eachframe when target objects determined to be the same object are detected,and derives a line obtained by linking the representative points betweenthese frames as a tracklet TL. Meanwhile, in a case where a plurality oftarget objects are detected in each of a plurality of frames in a resultof scanning the detection window W_(D), the object-tracker 112 mayderive a plurality of tracklets TL. The tracklet TL is assumed to show,for example, the moving direction of an object.

Next, the storage controller 114 superimposes one or more tracklets TLderived by the object-tracker 112 on one frame which is representative(for example, a frame of the latest time) among a plurality of framesreferred to when the tracklets TL are derived, and stores information(hereinafter, called tracking information) including the representativeframe on which the one or more tracklets TL are superimposed, in thestorage 130 (step S102). In addition, the storage controller 114 maycontrol the communicator 102 such that the tracking information isstored in an external storage device on a network.

Next, the output controller 116 causes the display 104 to display asetting screen for the detection line LN_(D) (step S104). The settingscreen for the detection line LN_(D) is, for example, a graphical userinterface (GUI) screen for accepting various types of input in order toset the detection line LN_(D). For example, the output controller 116may display a screen, as a GUI, which makes it possible to designatecoordinates of a start point and an end point of the detection lineLN_(D) on a frame on which tracklets TL included in the trackinginformation are superimposed.

Next, the detection region-setter 118 sets a detection line LN_(D) fordetecting passing through of a target object on the basis of thetracklet TL included in the tracking information, in a frame on whichtracklets TL are superimposed, in accordance with a user's operation ofthe operator 106 (step S106).

The detection line LN_(D) may be an approximative line obtained bylinking one or more pixels in an image (frame), and may be a line thatdoes not have a width geometrically defined as a function of a straightline or a curved line. In a case of the former, the detection lineLN_(D) may be a line having at least one pixel's worth of width.

For example, when a frame on which tracklets TL are superimposed isdisplayed as a setting screen on the display 104, a user may operate theoperator 106 while viewing this screen, and designate a plurality ofposition coordinates on the frame on which tracklets TL aresuperimposed. For example, in a case where coordinates of two points aredesignated by a user, the detection region-setter 118 may set a straightline connecting these two points as a detection line LN_(D). Inaddition, for example, in a case where coordinates of two or more pointsare designated by a user, the detection region-setter 118 may set acurve in which squares of distances from these points are a minimum, asa detection line LN_(D). The straight line or the curved line may be aone-dimensional line that does not have a width geometrically defined asa function, and may be a linear two-dimensional region in which severalpixels are formed together in a width direction.

In addition, the detection region-setter 118 may set a detection regionR_(D) which is a two-dimensional region instead of setting the detectionline LN_(D) of a straight line or a curved line. The detection regionR_(D) is, for example, a region which is set as a subspace(two-dimensional space) of a frame on which tracklets TL aresuperimposed, and is a space in which the length and breadth of a frameare represented as dimensions. For example, in a case where coordinatesof three or more points are designated by a user, the detectionregion-setter 118 may set a polygonal region obtained by linking eachpoint with a straight line, as a detection region R_(D), using eachpoint as a vertex. In addition, the detection region-setter 118 may seta stereoscopic three-dimensional detection region R_(D) in which eachpoint having coordinates designated by a user is used as a vertex. Inthe following description, as an example, a description will be given ofa case where the detection region-setter 118 sets a one-dimensionaldetection line LN_(D).

Next, the evaluation and analyzer 120 evaluates a setting mode for thedetection line LN_(D) with respect to each of one or more tracklets TLincluded in the tracking information, on the basis of the trackinginformation stored in the storage 130 or the external storage device andthe detection line LN_(D) set by the detection region-setter 118. In acase where the detection line LN_(D) which is a one-dimensionaldetection region R_(D) is set, the setting mode is, for example, theposition of the detection line LN_(D), the length of the detection lineLN_(D), the bending state of the detection line LN_(D), or the like. Inaddition, in a case where a two-or-more-dimensional detection regionR_(D) is set, the setting mode is, for example, the position of thedetection region R_(D), the area (volume) of the detection region R_(D),the shape of the detection region R_(D), or the like.

For example, the evaluation and analyzer 120 selects any one trackletTL, as a processing target, from among one or more tracklets TL includedin the tracking information, and determines whether this tracklet TLwhich is a target for processing and the detection line LN_(D) intersecteach other (step S108). The wording “intersect each other” means that,for example, the tracklet TL enters a pixel region including one or morepixels indicating the detection line LN_(D), and then leaves the pixelregion.

FIG. 4 is a diagram showing a method of determining the presence orabsence of intersection between the detection line LN_(D) and trackletsTL. For example, in a case where the detection line LN_(D) isrepresented as a line having a width equivalent to several pixels toseveral tens of pixels, the evaluation and analyzer 120 may set a circle(ellipse) passing through a start point and an end point of thedetection line LN_(D), and determines whether the tracklets TL and thedetection line LN_(D) intersect each other on the basis of two regions(regions A and B in the drawing) separated by the detection line LN_(D)among regions within this circle. For example, the evaluation andanalyzer 120 may determine a tracklet TL that does not enter from one ofthe regions separated into two parts by the detection line LN_(D) to theother region, within the set circle, like a tracklet TL1 or TL2, as atracklet TL that does not intersect the detection line LN_(D). Inaddition, the evaluation and analyzer 120 determines a tracklet TL thatenters from one of the regions separated into two parts by the detectionline LN_(D) to the other region, within the set circle, like a trackletTL3 or TL4, as a tracklet TL that intersects the detection line LN_(D).

In addition, in a case where the detection line LN_(D) is represented asa line having one pixel's worth of width, the evaluation and analyzer120 determines that the tracklet TL and the detection line LN_(D)intersect each other in a case where the fact that two pixels (forexample, adjacent pixels on the left side and the lower side) located ona diagonal line among four pixels which are vertically and horizontallyadjacent to a certain pixel to be focused upon among a plurality ofpixels indicating the detection line LN_(D) are pixels indicating thetracklet TL is established with respect to two or more pixels among aplurality of pixels indicating the detection line LN_(D).

FIGS. 5 and 6 are diagrams illustrating a method of determining thepresence or absence of intersection between the detection line LN_(D)and the tracklet TL. In the example of each drawing, the detection lineLN_(D) and the tracklet TL are represented as one pixel's worth of imageregion. Further, the detection line LN_(D) is represented by pixels A toH, and the tracklet TL is represented by pixels O to W. As illustratedin FIG. 5, in a case of being focusing on two pixels D and E among thepixels indicating the detection line LN_(D), an adjacent pixel locatedon the side above this pixel D focused upon and an adjacent pixellocated on the left side of the pixel E focused upon are set to thepixel R which is one of the pixels indicating the tracklet TL, and anadjacent pixel located on the right side of the pixel D focused upon andan adjacent pixel located on the side below the pixel E focused upon areset to the pixel S which is one of the pixels indicating the trackletTL. In such a case, the evaluation and analyzer 120 determines that thedetection line LN_(D) and the tracklet TL intersect each other.

On the other hand, as illustrated in FIG. 6, in a case of being focusingon two pixels D and E among the pixels indicating the detection lineLN_(D), an adjacent pixel located on the side above this pixel D focusedupon and an adjacent pixel located on the left side of the pixel Efocused upon are set to the pixel R which is one of the pixelsindicating the tracklet TL, but adjacent pixels other than the pixel Dfocused upon are not set to pixels indicating the tracklet TL. In such acase, the evaluation and analyzer 120 determines that the detection lineLN_(D) and the tracklet TL do not intersect each other.

The evaluation and analyzer 120 makes different evaluations of thesetting mode for the detection line LN_(D) in a case where the trackletTL and the detection line LN_(D) intersect each other and a case wherethe tracklet TL and the detection line LN_(D) do not intersect eachother. For example, in a case where the tracklet TL and the detectionline LN_(D) intersect each other, the evaluation and analyzer 120 mayevaluate a higher setting mode for the detection line LN_(D) than in acase where the tracklet TL and the detection line LN_(D) do notintersect each other.

The output controller 116 changes a display mode for the trackinginformation on the basis of the evaluation result of the evaluation andanalyzer 120. For example, since evaluations are different from eachother in a case where the tracklet TL and the detection line LN_(D)intersect each other and a case where the tracklet TL and the detectionline LN_(D) do not intersect each other, the output controller 116 makesthe display modes for the detection line LN_(D) different from eachother in these respective cases.

For example, in a case where it is determined by the evaluation andanalyzer 120 that the tracklet TL and the detection line LN_(D)intersect each other, the output controller 116 may cause the display104 to highlight a tracklet TL intersecting the detection line LN_(D)(step S110). The highlighting may be to change, for example, the displaycolor of the tracklet TL to other colors. In addition, for example,instead of or in addition to changing the color of the tracklet TL, thehighlighting may be to further increase an element value such asbrightness, chroma, or luminance, may be to further increase the linethickness of the tracklet TL, may be to decrease the line transparencyof the tracklet TL, and may be to change the texture of the tracklet TL.

On the other hand, in a case where it is determined by the evaluationand analyzer 120 that the tracklet TL and the detection line LN_(D) donot intersect each other, the output controller 116 causes the display104 to lowlight a tracklet TL not intersecting the detection line LN_(D)(step S112). The lowlighting is to, for example, maintain the currentcolor without changing the display color of the tracklet TL to othercolors. In addition, the lowlighting may be to, for example, reduce anelement value such as brightness, chroma, or luminance of the displaycolor of the tracklet TL compared with during highlighting, may be toreduce the line thickness of the tracklet TL compared with duringhighlighting, and may be to increase the line transparency of thetracklet TL compared with during highlighting.

FIGS. 7 to 10 are diagrams illustrating a method of displaying trackletsTL. In the example of FIG. 7, among four tracklets TL, tracklets TL1,TL2, and TL3 intersects the detection line LN_(D), and a tracklet TL4does not intersect the detection line LN_(D). In this case, for example,the output controller 116 may cause the display 104 to highlight thetracklets TL1, TL2, and TL3 by making their display colors red, and tolowlight the tracklet TL4 by making its display color gray.

In addition, the output controller 116 may cause the display 104 todisplay the tracklets TL as a heat map as shown in FIG. 8, and may causethe display 104 to display a numerical value indicating the ratio of thenumber of tracklets TL intersecting the detection line LN_(D) to thetotal number of tracklets TL as shown in FIG. 9.

In addition, the output controller 116 may change display modes inaccordance with the directions of the tracklets TL. For example, asshown in FIG. 10, the output controller 116 may highlight each of thetracklets TL by making display colors red with respect to tracklets TLof which the moving directions (directions of arrows in the drawing) areinto the entrance port, and making display colors blue with respect totracklets TL of which the moving directions are out from the entranceport.

The controller 110 terminates the processing of the present flow diagramin a case where intersection with the detection line LN_(D) isdetermined with respect to all the tracklets TL, changes tracklets TLwhich are targets for processing in a case where intersection with thedetection line LN_(D) is not determined with respect to all thetracklets TL, and repeats the processes of S108 to S112.

Meanwhile, in the processing of the flow diagram described above, theevaluation and analyzer 120 may determine whether the tracklet TL andthe detection region R_(D) intersect each other in a case where atwo-dimensional or three-dimensional detection region R_(D) is set bythe detection region-setter 118 instead of the one-dimensional detectionline LN_(D). The term “intersect” refers to, for example, a positionalrelationship in which the tracklet TL and the detection region R_(D)intrude into each other.

According to the first embodiment described above, a target object istracked in a moving image, tracking information including a tracklet TLwhich is a movement trajectory of the tracked object is stored in thestorage 130 or the external storage device, a detection line LN_(D) or adetection region R_(D) for detecting passing through of an object is setin a frame within the moving image on the basis of the tracklet TLincluded in the tracking information, a setting mode for the detectionline LN_(D) or the detection region R_(D) is evaluated on the basis ofthis set detection line LN_(D) or detection region R_(D) and thetracklet TL, and the tracking information is output in an output modebased on the evaluation result, whereby it is possible to improve theaccuracy of detection of an entering or leaving person.

For example, in a technique disclosed in Patent Document 1, a user ispresented with a recommended region for setting a detection line LN_(D)with reference to the past person's tracklet TL. However, in a casewhere there is a person passing through a place other than an entranceport such as, for example, a person passing by the entrance port, thereis a problem in that it may not be possible to present a correctrecommended region, which leads to a decrease in the accuracy ofdetection of an object. In addition, in a case of a technique disclosedin Patent Document 2, generally, a screen end region in which there is adecrease in the accuracy of detection of a tracklet TL of a trackedperson is set to a setting forbidden region. However, a problem occursin that a situation in which there is a person passing through a placeother than an entrance port can occur even in a region at the center ofa screen, which leads to a decrease in the accuracy of detection of anobject.

On the other hand, in the present embodiment, it is possible to cause auser to confirm whether a past tracklet TL can be correctly detectedusing the detection line LN_(D) or the detection region R_(D) set inaccordance with the user's operation, and thus it is possible to causethe user to set the detection line LN_(D) or the like again, forexample, so as not to intersect the tracklet TL of a person passingthrough a place other than an entrance port. As a result, even in a casewhere there is a person passing through a place other than an entranceport, it is possible to detect a person who enters an entrance port orleaves the entrance port with a good degree of accuracy.

Second Embodiment

Hereinafter, a second embodiment will be described. The secondembodiment is different from the first embodiment in that it isdetermined whether tracklets TL selected by a user among a plurality oftracklets TL intersect the detection line LN_(D) or the detection regionR_(D), and a setting mode for the detection line LN_(D) or the detectionregion R_(D) is evaluated for each of the tracklets TL selected by theuser, on the basis of the determination result. Hereinafter, adescription will be given with focus on differences from the firstembodiment, and common parts with respect to those in the firstembodiment will not be described.

FIG. 11 is a flow diagram showing a flow of a series of processesperformed by a controller 110 of the second embodiment. Processing ofthe present flow diagram may be repeated, for example, whenever a movingimage is acquired from the camera 10.

First, the object-tracker 112 tracks a target object in a moving imageacquired from the camera 10 (step S200). Next, the storage controller114 superimposes one or more tracklets TL derived by the object-tracker112 in order to track a target object, on one frame which isrepresentative among a plurality of frames referred to when thetracklets TL are derived, and cause the storage 130 or the like to storetracking information including the representative frame on which the oneor more tracklets TL are superimposed (step S202).

Next, the output controller 116 causes the display 104 to display aselection screen for tracklets TL (step S204). The selection screen fortracklets TL is, for example, a screen for causing a user to select atracklet TL (tracklet TL which is a target for detection) desired to bedetected using the detection line LN_(D) or the detection region R_(D)from among all the tracklets TL derived by the object-tracker 112.

FIG. 12 is a diagram showing an example of a selection screen fortracklets TL. In a case where the screen of the shown example isdisplayed, a user operates the operator 106, and selects one or moretracklet TL, as tracklets TL which are targets for detection, from amongfive tracklets TL.

Next, the output controller 116 causes the display 104 to display theselection screen for tracklets TL followed by determining whether one ormore tracklets TL have been selected by a user (step S206), and causesthe display 104 to display a setting screen for the detection lineLN_(D) in a case where one or more tracklets TL are selected by the user(step S208).

FIG. 13 is a diagram showing an example of a setting screen for thedetection line LN_(D). In the shown example, tracklets TL1, TL2, and TL3are selected as tracklets TL which are targets for detection, and theremaining tracklets TL4 and TL5 which are not selected are set totracklets TL which are targets for non-detection. In this case, theoutput controller 116 may make display modes for the tracklets TL whichare targets for detection and the tracklets TL which are targets fornon-detection different from each other. For example, the outputcontroller 116 may make the display color of the tracklets TL which aretargets for detection red, and make the display color of the trackletsTL which are targets for non-detection gray. Thereby, a user can set thedetection line LN_(D) while confirming which tracklets TL the detectionline may be caused to intersect.

Next, the evaluation and analyzer 120 selects any one tracklet TL, as aprocessing target, from among one or more tracklets TL included in thetracking information, and determines whether this tracklet TL which is atarget for processing and the detection line LN_(D) intersect each other(step S212).

Next, in a case where it is determined that the tracklet TL which is atarget for processing and the detection line LN_(D) intersect eachother, the evaluation and analyzer 120 further determines whether thetracklet TL which is a target for processing is a tracklet TL selectedas a detection target by a user (step S214).

On the other hand, in a case where it is determined that the tracklet TLwhich is a target for processing and the detection line LN_(D) do notintersect each other, the evaluation and analyzer 120 further determineswhether the tracklet TL which is a target for processing is a trackletTL selected as a detection target by a user (step S216).

The evaluation and analyzer 120 evaluates the setting mode for thedetection line LN_(D) for each tracklet TL, on the basis of thesevarious types of determination result.

For example, the evaluation and analyzer 120 may evaluate a high settingmode for the detection line LN_(D) with respect to tracklets TL thatintersect the detection line LN_(D) among one or more tracklets TLselected as detection targets by a user, and evaluate a low setting modefor the detection line LN_(D) with respect to tracklets TL that do notintersect the detection line LN_(D).

In addition, for example, the evaluation and analyzer 120 evaluates alow setting mode for the detection line LN_(D) with respect to trackletsTL that intersect the detection line LN_(D) among one or more trackletsTL which are targets for non-detection and are not selected as detectiontargets by a user, and evaluates a high setting mode for the detectionline LN_(D) with respect to tracklets TL that do not intersect thedetection line LN_(D).

FIG. 14 is a diagram showing an example of a screen on which thedetection line LN_(D) is set. In the shown example, the set detectionline LN_(D) is drawn with respect to frames in which the tracklets TL1,TL2, and TL3 are selected as tracklets TL which are targets fordetection. The tracklets TL2 and TL3 which are targets for detectionintersect this set detection line LN_(D), and the tracklet TL4 which isa target for non-detection intersects the set detection line. That is,the tracklets TL2 and TL3 which are targets for detection are in a“correctly detected state” which is being detected by the detection lineLN_(D) as desired by a user, and the tracklet TL4 which is a target fornon-detection is in an “over-detected state” which is being detected bythe detection line LN_(D) in spite of this not being desired by a user.

In addition, the tracklet TL1 which is a target for detection does notintersect the detection line LN_(D), and the tracklet TL5 which is atarget for non-detection does not intersect the detection line. That is,the tracklet TL1 which is a target for detection is in a “non-detectedstate” which is not being detected by the detection line LN_(D) in spiteof this being desired by a user, and the tracklet TL5 which is a targetfor non-detection is in a “correctly non-detected state” which is notbeing detected by the detection line LN_(D) as desired by a user.

In such a case, the evaluation and analyzer 120 evaluates a high settingmode for the detection line LN_(D) with respect to the tracklets TL2 andTL3 which are targets for detection in the “correctly detected state”and the tracklet TL5 which is a target for non-detection in the“correctly non-detected state”, and evaluates a low setting mode for thedetection line LN_(D) with respect to the tracklet TL1 which is a targetfor detection in the “non-detected state” and the tracklet TL4 which isa target for non-detection in the “over-detected state”.

The output controller 116 changes the display modes for tracklets TL onthe basis of the evaluation results of the setting modes for thedetection line LN_(D) with respect to these respective tracklets TL.

For example, in a case where it is determined by the evaluation andanalyzer 120 that the tracklet TL which is a target for processingintersects the detection line LN_(D), and is a tracklet TL which is atarget for detection, that is, in a case of being in the “correctlydetected state”, the output controller 116 causes the display 104 toperform first highlighting on the tracklet TL which is a target forprocessing (step S218). The first highlighting refers to, for example,greater highlighting than in other display modes such as secondhighlighting or third highlighting described later.

In addition, in a case where it is determined by the evaluation andanalyzer 120 that the tracklet TL which is a target for processingintersects detection line LN_(D), and is not a tracklet TL which is atarget for detection, that is, in a case of being in a “misdetectedstate”, the output controller 116 causes the display 104 to performsecond highlighting on the tracklet TL which is a target for processing(step S220). The second highlighting refers to, for example, the samedegree of highlighting as third highlighting, or greater highlightingthan the third highlighting.

In addition, in a case where it is determined by the evaluation andanalyzer 120 that the tracklet TL which is a target for processing doesnot intersect the detection line LN_(D), and is a tracklet TL which is atarget for detection, that is, in a case of being in the “non-detectedstate”, the output controller 116 causes the display 104 to performthird highlighting on the tracklet TL which is a target for processing(step S222). The third highlighting refers to, for example, greaterhighlighting than at least lowlighting.

In addition, in a case where it is determined by the evaluation andanalyzer 120 that the tracklet TL which is a target for processing doesnot intersect the detection line LN_(D), and is not a tracklet TL whichis a target for detection, that is, in a case of being in the “correctlynon-detected state”, the output controller 116 causes the display 104 tolowlight the tracklet TL which is a target for processing (step S224).

FIG. 15 is a diagram showing an example of tracklets TL displayed in adisplay mode based on each evaluation result. As in the shown example,the output controller 116 may determine display modes (such as, forexample, color or texture) for tracklets TL in accordance with eachevaluation result of the detection line LN_(D) with respect to eachtracklet TL such as a “correctly detected state”, a “misdetected state”,a “non-detected state”, and a “correctly non-detected state”.

The controller 110 terminates the processing of the present flow diagramin a case where intersection with the detection line LN_(D) isdetermined with respect to all the tracklets TL, changes tracklets TLwhich are targets for processing in a case where intersection with thedetection line LN_(D) is not determined with respect to all thetracklets TL, and repeats the processes of S212 to S224.

According to the second embodiment described above, it is determinedwhether tracklets TL selected by a user among a plurality of trackletsTL intersect the detection line LN_(D) or the detection region R_(D),and a setting mode for the detection line LN_(D) or the detection regionR_(D) is evaluated for each of the tracklets TL selected by the user, onthe basis of the determination result. Therefore, it is possible topresent to a user whether a tracklet TL of a person tracked in the pastcan be correctly detected or is erroneously detected with respect to thedetection line LN_(D) or the like tentatively set by the user, andpossible for the user himself (or herself) to determine whether the setdetection line LN_(D) or the like operates as intended by the user.

Third Embodiment

Hereinafter, a third embodiment will be described. The third embodimentis different from the first and second embodiments in that when thetracklet TL and the detection line LN_(D) intersect each other, thesetting mode for the detection line LN_(D) is evaluated in accordancewith an intersection position on the tracklet TL, an intersectionposition on the detection line LN_(D), and some or all of the angles(included angles) between the tracklet TL and the detection line LN_(D).Hereinafter, a description will be given with focus on differences fromthe first and second embodiments, and common parts with respect to thosein the first and second embodiments will not be described.

FIGS. 16 to 18 are diagrams illustrating an evaluation method in a casewhere the tracklet TL and the detection line LN_(D) intersect eachother. For example, an end farthest from the midpoint of the tracklet TLis a position at which the tracking of a target object is interrupted.In a case where the detection line LN_(D) is set in the vicinity of thisend, a target object which is being tracked may be interrupted withoutpassing through the detection line LN_(D). Therefore, in order to stablydetect passing through of a target object, it is preferable that thedetection line LN_(D) be set in the vicinity of the midpoint of thetracklet TL. For this reason, for example, in the tracklet TL, theevaluation and analyzer 120 of the third embodiment may increase a scoreobtained by quantifying the degree of evaluation as the point ofintersection with the detection line LN_(D) approaches the center(midpoint) of a length D_(TL) of the tracklet TL, and may lower thescore as the point of intersection with the detection line LN_(D)becomes farther away from the center of the length D_(TL) of thetracklet TL.

In addition, in a case where the end of the detection line LN_(D) andthe tracklet TL are set so as to intersect each other, a target objectwhich is being tracked travels away from the detection line LN_(D), andthus the target object may not be able to be detected. Therefore, inorder to stably detect passing through of a target object, it ispreferable that the vicinity of the midpoint of the detection lineLN_(D) and the tracklet TL be set so as to intersect each other. Forthis reason, for example, in the detection line LN_(D), the evaluationand analyzer 120 may increase the score as the point of intersectionwith the tracklet TL approaches the center (midpoint) of a lengthD_(LND) of the detection line LN_(D), and may lower the score as thepoint of intersection with the tracklet TL becomes farther away from thecenter of the length D_(LND) of the detection line LN_(D).

In addition, in a case where the detection line LN_(D) is set so as tobe parallel to the tracklet TL, there is a decreasing probability of atarget object which is being tracked passing through the detection lineLN_(D), and thus it is preferable that the detection line LN_(D) be setso as to be orthogonal to the tracklet TL. For this reason, theevaluation and analyzer 120 may increase the score, for example, as anangle θ between the tracklet TL and the detection line LN_(D) approaches90 degrees, and may lower the score as the angle θ approaches 0 degreesor 180 degrees.

The evaluation and analyzer 120 evaluates a setting mode for a finaldetection line LN_(D) on the basis of some or all of a score(hereinafter, called a first score) obtained by evaluating the positionof the point of intersection with the detection line LN_(D) on thetracklet TL, a score (hereinafter, called a second score) obtained byevaluating the position of the point of intersection with the trackletTL on the detection line LN_(D), and a score (hereinafter, called athird score) obtained by evaluating the angle θ between the tracklet TLand the detection line LN_(D). For example, in a case where all thescores are referred to, the evaluation and analyzer 120 may evaluate ahighest setting mode in which a weighted sum of these scores becomesmaximum. In this case, the evaluation and analyzer 120 evaluates ahighest setting mode for the detection line LN_(D) set by a user in acase where the center of the tracklet TL and the center of the detectionline LN_(D) intersect each other in a state where the tracklet TL andthe detection line LN_(D) are orthogonal to each other.

FIGS. 19 to 22 are diagrams illustrating an example of a setting modefor the detection line LN_(D). For example, as shown in FIG. 19, in acase where the detection line LN_(D) is set at a position close to theend of the tracklet TL, there is an increasing probability of thetracklet TL of another target object and the detection line LN_(D) notintersecting each other, and thus the evaluation and analyzer 120evaluates a lower setting mode for this detection line LN_(D) than asetting mode for the detection line LN_(D) as illustrated in FIG. 22. Inthis case, the output controller 116 causes, for example, the display104 to display a blue detection line LN_(D) in order to indicate thatthe accuracy of detection of a target object based on the detection lineLN_(D) is low, that is, to indicate that the probability ofnon-detection increases.

In addition, for example, as shown in FIG. 20, in a case where thedetection line LN_(D) is set in a nearly parallel state such that anangle θ between the detection line and the tracklet TL is 150 degrees,there is a decreasing probability of a target object passing through thedetection line LN_(D), and thus the evaluation and analyzer 120evaluates a lower setting mode for this detection line LN_(D) than asetting mode for the detection line LN_(D) as illustrated in FIG. 22. Inthis case, similarly to the example of FIG. 19, the output controller116 causes the display 104 to display a blue detection line LN_(D).

In addition, for example, as shown in FIG. 21, in a case where thevicinity of the end of the detection line LN_(D) is set so as tointersect the tracklet TL, there is a decreasing probability of a targetobject passing through the detection line LN_(D), and thus theevaluation and analyzer 120 evaluates a lower setting mode for thisdetection line LN_(D) than a setting mode for the detection line LN_(D)as illustrated in FIG. 22. In this case, similarly to the example ofFIG. 19 or 20, the output controller 116 causes the display 104 todisplay a blue detection line LN_(D).

In addition, for example, as shown in FIG. 22, in a case where thevicinity of the center of the detection line LN_(D) is set so as tointersect the vicinity of the center of the tracklet TL in a state wherethe tracklet TL and the detection line LN_(D) are nearly orthogonal toeach other, there is an increasing probability of a target objectpassing through the detection line LN_(D), and thus the evaluation andanalyzer 120 evaluates a higher setting mode for this detection lineLN_(D) than the setting mode for the detection line LN_(D) asillustrated in FIGS. 19 to 21. In this case, the output controller 116causes, for example, the display 104 to display a yellow detection lineLN_(D) in order to indicate that the accuracy of detection of a targetobject based on the detection line LN_(D) is high, that is, to indicatethat the probability of non-detection has decreased.

Meanwhile, in the above-described example, the representation of thedegree of the accuracy of detection of a target object by the color ofthe detection line LN_(D) is not limited thereto. For example, theoutput controller 116 may represent the degree of the accuracy ofdetection of a target object, by the transparency, texture, linethickness or the like of the detection line LN_(D), and may displaycharacters or images indicating that the accuracy of detection of atarget object is high or low, separately from the detection line LN_(D).In addition, the output controller 116 may output the high or lowaccuracy of detection of a target object, as sounds, from a speaker (notshown) or the like. In addition, the output controller 116 may displaythe value of a weighted sum of the first score, the second score and thethird score, as a probability value indicating the accuracy of detectionof a target object.

According to the third embodiment described above, the tracklet TL andthe detection line LN_(D) intersect each other, but in a case where thetracklet TL is shortened without being able to sufficiently track atarget object, or the like, a display method different from usual isused in a detection line LN_(D) determined to have a low probability ofthe target object being able to be correctly detected, whereby it ispossible for a user to confirm how accurately the set detection lineLN_(D) has been detected. Thereby, for example, since a user resets thedetection line LN_(D) again, it is possible to set the detection lineLN_(D) which makes it possible to detect a target object with a higherdegree of accuracy.

Fourth Embodiment

Hereinafter, a fourth embodiment will be described. The fourthembodiment is different from the first to third embodiments in that whenthe tracklet TL and the detection line LN_(D) intersect each other, thesetting mode for the detection line LN_(D) is evaluated in accordancewith one or both of a distance from another tracklet TL and an anglebetween another tracklet TL and the detection line. Hereinafter, adescription will be given with focus on differences from the first tothird embodiments, and common parts with respect to those in the firstto third embodiments will not be described.

For example, as is the case with the second embodiment, in a case wherea tracklet TL selected as a detection target by a user and a tracklet TLwhich is a target for non-detection, not selected as a detection targetby the user, are set among a plurality of tracklets TL, the detectionline LN_(D) may intersect the tracklet TL which is a target fordetection, and may not intersect the tracklet TL which is a target fornon-detection. That is, depending on the setting position of thedetection line LN_(D), as desired by a user, the tracklet TL which is atarget for detection may be set to be in a “correctly detected state”,and the tracklet TL which is a target for non-detection may be set to bein a “correctly non-detected state”. However, in a case where thedetection line LN_(D) is set at a position close to the tracklet TLwhich is a target for non-detection, or the extended line of thedetection line LN_(D) is set so as to intersect the tracklet TL which isa target for non-detection, there is an increasing probability of eventhe tracklet TL which is a target for non-detection being erroneouslydetected by the detection line LN_(D).

Therefore, even in a case where the detection line LN_(D) does notintersect the tracklet TL which is a target for non-detection, and ahigh evaluation of the setting mode is made, an evaluation and analyzer120 of the fourth embodiment lowers the evaluation of the setting modefor the detection line LN_(D) in a case where a distance between thetracklet TL and the detection line LN_(D) are short, or a case where theextended line of the detection line LN_(D) intersects the tracklet TLwhich is a target for non-detection at a nearly orthogonal angle.

For example, when the tracklet TL which is a target for detection andthe detection line LN_(D) intersect each other, the evaluation andanalyzer 120 may increase the score as a distance between (shortestdistance) the tracklet TL which is a target for non-detection and thedetection line LN_(D) becomes longer, and may lower the score as thedistance between the tracklet TL which is a target for non-detection andthe detection line LN_(D) becomes shorter.

In addition, for example, when the tracklet TL which is a target fordetection and the detection line LN_(D) intersect each other, theevaluation and analyzer 120 may increase the score as an angle φ betweenthe tracklet TL which is a target for non-detection and the detectionline LN_(D) approaches 0 degrees or 180 degrees, and may lower the scoreas the angle φ approaches 90 degrees.

The evaluation and analyzer 120 evaluates a setting mode for a finaldetection line LN_(D) on the basis of one or both of a score(hereinafter, called a fourth score) obtained by evaluating a distancebetween the tracklet TL which is a target for non-detection and thedetection line LN_(D) and a score (hereinafter, called a fifth score)obtained by evaluating an angle φ between the tracklet TL which is atarget for non-detection and the detection line LN_(D). For example, theevaluation and analyzer 120 evaluates a higher setting mode as aweighted sum of these scores becomes larger.

FIGS. 23 and 24 are diagrams illustrating another example of the settingmode for the detection line LN_(D). In the drawing, TL1 is a trackletwhich is a target for detection, and TL2 is a tracklet which is a targetfor non-detection. In the example of FIG. 23, since the detection lineLN_(D) and the tracklet TL1 which is a target for detection intersecteach other in a nearly orthogonal state, the evaluation and analyzer 120originally evaluates a high setting mode for this detection line LN_(D).However, since a distance between the detection line LN_(D) and thetracklet TL2 which is a target for non-detection is short, theevaluation and analyzer 120 evaluates a lower setting mode than thesetting mode for the detection line LN_(D) as illustrated in FIG. 22described above. In this case, the output controller 116 causes, forexample, the display 104 to display a red detection line LN_(D) in orderto indicate that the accuracy of detection of a target object based onthe detection line LN_(D) is low, that is, to indicate that theprobability of over-detection increases.

In addition, in the example of FIG. 24, the detection line LN_(D) doesnot intersect the tracklet TL2 which is a target for non-detection,while the extended line of the detection line LN_(D) intersects thetracklet. In such a case, the evaluation and analyzer 120 evaluates alower setting mode than the setting mode for the detection line LN_(D)as illustrated in FIG. 22 described above. The output controller 116causes, for example, the display 104 to display a red detection lineLN_(D) in order to indicate that the accuracy of detection of a targetobject based on the detection line LN_(D) is low, that is, to indicatethat the probability of over-detection increases.

Meanwhile, the evaluation and analyzer 120 may evaluate the setting modefor the detection line LN_(D) in further consideration of some or all ofthe intersection position on the tracklet TL, the intersection positionon the detection line LN_(D), and the angle between the tracklet TL andthe detection line LN_(D) which are described in the aforementionedthird embodiment, in addition to one or both of a distance from thetracklet TL which is a target for non-detection and an angle between thetracklet TL which is a target for non-detection and the detection line.In a case where these five elements are considered, the evaluation andanalyzer 120 may evaluate a higher setting mode for the detection lineLN_(D) as a weighted sum of all the first to fifth scores becomeslarger.

According to the fourth embodiment described above, in a case where thetracklet TL which is a target for detection and the tracklet TL which isa target for non-detection are designated by a user, the tracklet TLwhich is a target for detection and the detection line LN_(D) intersecteach other, and the tracklet TL which is a target for non-detection andthe detection line LN_(D) do not intersect each other, but a displaymethod different from usual is used in a detection line LN_(D)(detection line LN_(D) having the fourth score and the fifth score whichare low) having a high probability of erroneously detecting passingthrough of a target object depicting the same movement trajectory asthat of the tracklet TL which is a target for non-detection, whereby itis possible for a user to confirm how accurately the set detection lineLN_(D) has been detected. Thereby, for example, since a user resets thedetection line LN_(D) again, it is possible to set the detection lineLN_(D) which makes it possible to detect a target object with a higherdegree of accuracy.

Fifth Embodiment

Hereinafter, a fifth embodiment will be described. The fifth embodimentis different from the first to fourth embodiments in that candidates forthe detection line LN_(D) are presented in advance, and a user is causedto select a candidate used as the detection line LN_(D) from among thesecandidates. Hereinafter, a description will be given with focus ondifferences from the first to fourth embodiments, and common parts withrespect to those in the first to fourth embodiments will not bedescribed.

A detection region-setter 118 of the fifth embodiment sets, for example,a straight line or a curved line, having the most number ofintersections with tracklets TL included in a certain first group, andthe least number of intersections with tracklets TL included in a secondgroup which is a set of tracklets TL excluding the first group, as acandidate for the detection line LN_(D). The first group is a set of oneor more tracklets TL of which start points or end points are located inthe vicinity of an entrance port.

FIG. 25 is a diagram showing a method of setting candidates for thedetection line LN_(D). Tracklets TL1 to TL3 in the drawing are includedin the first group, and tracklets TL4 and TL5 are included in the secondgroup. In this case, the detection region-setter 118 sets, for example,a candidate LN_(D)#1 or LN_(D)#2 which is a detection line. Meanwhile,the detection region-setter 118 may set a candidate for a detectionline, such as LN_(D)#3, which intersects a tracklet TL which is a partof the second group.

In addition, the detection region-setter 118 may set, for example, astraight line or a curved line, having the maximum number ofintersections with tracklets TL facing toward the direction of theentrance port or toward the opposite direction of the entrance port, andthe minimum number of intersections with other tracklets TL, as acandidate for the detection line LN_(D).

In addition, the detection region-setter 118 may set, for example, astraight line or a curved line, having the maximum number ofintersections with tracklets TL selected as detection targets by a user,and the minimum number of intersections with other tracklets TL, as acandidate for the detection line LN_(D).

The detection region-setter 118 determines a candidate selected fromamong candidates for one or more detection lines LN_(D) presented to auser, as the detection line LN_(D). Meanwhile, in a case where the startpoint, the end point or the like of a straight line or a curved linedetermined as the detection line LN_(D) from among candidates fordetection lines LN_(D) is re-designated by a user, the detectionregion-setter 118 may change the length, the degree of curvature or thelike of the detection line LN_(D) so as to pass through thisre-designated point.

In addition, the detection region-setter 118 of the fifth embodiment maynewly set candidates for one or more detection lines LN_(D) on the basisof the evaluation of a setting mode for the detection line LN_(D) set inthe past in accordance with a user's operation. For example, a pluralityof detection lines LN_(D) having different evaluations of setting modesare set with respect to the patterns of tracklets TL such as a relativepositional relationship between a plurality of tracklets TL, and thelength or the degree of curvature of each of a plurality of trackletsTL. In this case, in a case where the pattern of a tracklet TL in thistime is similar to the pattern of a tracklet TL in the past whencandidates for detection lines LN_(D) are set, the detectionregion-setter 118 preferentially sets a detection line LN_(D) having ahigher evaluation of a setting mode among a plurality of detection linesLN_(D) set at the tracklet TL of this pattern, as a candidate for adetection line LN_(D) in this time.

In addition, the detection region-setter 118 of the fifth embodiment maylearn a user's preferences for a setting mode for the detection lineLN_(D), and set the detection line LN_(D) automatically (irrespective ofa user's operation) in accordance with the learned preference. Forexample, when a user sets a tracklet TL which is a target for detectionand a tracklet TL which is a target for non-detection, in a case where adetection line LN_(D) is often set at a position where the tracklet TLwhich is a target for non-detection is detected together with thetracklet TL which is a target for detection, that is, a case where adetection line LN_(D) having a certain amount of “over-detection”allowed is often set in order to prioritize “correct detection”, thedetection region-setter 118 learns that this user prefers to improve theaccuracy of detection of a tracklet TL by, forcibly if necessary,allowing the over-detection. In addition, for example, when a user setsa tracklet TL which is a target for detection and a tracklet TL which isa target for non-detection, in a case where a detection line LN_(D) isoften set at a preferential position where the tracklet TL which is atarget for non-detection is not detected though the tracklet TL which isa target for detection is not detected, that is, a case where adetection line LN_(D) having a certain amount of “non-detection” allowedis often set in order to prioritize not to be “over-detected”, thedetection region-setter 118 learns that this user prefers to prohibitthe over-detection by, forcibly if necessary, lowering the accuracy ofdetection of a tracklet TL.

The detection region-setter 118 automatically sets a new detection lineLN_(D) in accordance with this learned user's preferences for a settingmode for the detection line LN_(D). For example, in a case where a userhaving learned a tendency to set a detection line LN_(D) having acertain amount of “over-detection” allowed in order to prioritize“correct detection” attempts to newly set a detection line LN_(D), thedetection region-setter 118 may automatically set a detection lineLN_(D) which is given the highest-priority of “correct detection” whileallowing the “over-detection”, on the basis of this learning result.Thereby, it is possible to save much time and effort for a user to set adetection line LN_(D), and to improve the user's convenience.

Meanwhile, a user's preferences for a setting mode for the detectionline LN_(D) which are learned by the detection region-setter 118 may bedetermined by the user himself (or herself). FIG. 26 is a diagramshowing an example of a screen for setting the preferences for a settingmode for the detection line LN_(D). For example, the output controller116 causes the display 104 to display a screen as illustrated in FIG.26, and causes a user to set the preferences for a setting mode for thedetection line LN_(D). A bar indicating a threshold in the drawing isused for adjusting, for example, the degree of the accuracy of detectionof the tracklet TL based on the detection line LN_(D). A user moves thisbar to the 0[%] side or the 100[%] side, to thereby make a setting to adesired accuracy of detection. The 0[%] side indicates, for example,that the tracklet TL is prohibited from being set to “over-detection” bythe detection line LN_(D), and the 100[%] side indicates, for example,that the tracklet TL is prohibited from being set to “non-detection” bythe detection line LN_(D).

In addition, in a case where the detection line LN_(D) is set for eacharea under surveillance of each camera 10, the detection region-setter118 may set a setting mode for the detection line LN_(D) set withrespect to a certain area under surveillance, to a setting mode for thedetection line LN_(D) set with respect to another area undersurveillance.

FIG. 27 is a diagram showing a method of reflecting a setting mode forthe detection line LN_(D) of a certain area under surveillance in asetting mode for the detection line LN_(D) of another area undersurveillance. For example, in a case where one hundred cameras 10 areinstalled in a certain facility X, a user who manages the facility X mayset the detection line LN_(D) at an update timing determined in advance,with respect to the area under surveillance of each of the one hundredcameras 10. In this case, the detection region-setter 118 may set adetection line LN_(D) in another area under surveillance such as an areaunder surveillance B or C, in a setting mode for the detection lineLN_(D) set with respect to a certain area under surveillance A. Thereby,it is possible to save more time and effort than when a user setsdetection lines LN_(D) with respect to all the areas under surveillance,and to improve the user's convenience.

According to the fifth embodiment described above, since a user ispresented candidates for detection lines LN_(D) which are set in advanceso as to pass through a certain amount of tracklets TL, the user mayjust select a detection line LN_(D) from among the presented candidates,and it is possible to save much time and effort for the user to set thedetection line LN_(D). As a result, it is possible to improve a user'sconvenience.

In addition, according to the fifth embodiment described above, sincecandidates for one or more detection lines LN_(D) are newly set on thebasis of the evaluation of a setting mode for the detection line LN_(D)set in the past in accordance with a user's operation, a detection lineLN_(D) close to a mode when the user performs a setting by himself (orherself) can be selected from among candidates for a plurality ofdetection lines LN_(D). As a result, it is possible to further save muchtime and effort for the user to set the detection line LN_(D), and toimprove the user's convenience.

In addition, according to the fifth embodiment described above, since auser' preferences for a setting mode for the detection line LN_(D) arelearned, it is possible to automatically set the detection line LN_(D)in accordance with the user's preference.

In addition, according to the fifth embodiment described above, sincethe detection line LN_(D) is uniformly set in each area undersurveillance for each camera 10 on the basis of the learned user'spreferences for a setting mode for the detection line LN_(D), it ispossible to save more time and effort than when a user sets detectionlines LN_(D) with respect to all the areas under surveillance, and toimprove the user's convenience.

Sixth Embodiment

Hereinafter, a sixth embodiment will be described. The sixth embodimentis different from the first to fifth embodiments in that when an objectis tracked in a moving image, the degree of conviction indicating thedegree of probability of the tracked object being a target object isobtained, and tracking information including only a tracklet TL which isthe movement trajectory of an object of which the degree of convictionis equal to or greater than a threshold is stored in the storage 130 orthe external storage device. Hereinafter, a description will be givenwith focus on differences from the first to fifth embodiments, andcommon parts with respect to those in the first to fifth embodimentswill not be described.

An object-tracker 112 of the sixth embodiment derives the degree ofconviction for each object which is a target for tracking. For example,the object-tracker 112 may derive a similarity between an image regionoverlapping a detection window W_(D) and a template image as the degreeof conviction, may derive a similarity between objects detected in eachof a plurality of frames as the degree of conviction, may derive asimilarity between the image region overlapping the detection windowW_(D) and a background image as the degree of conviction, and may derivea difference in a hue, a luminance value, a luminance gradient or thelike between a plurality of frames as the degree of conviction. Inaddition, the object-tracker 112 may derive a value such as a weightedsum when these parameters are combined as the degree of conviction.

A storage controller 114 of the sixth embodiment sets an object of whichthe degree of conviction is equal to or greater than a threshold, amongobjects of which the degrees of conviction are derived by theobject-tracker 112, to a target object (for example, person), and causesthe storage 130 or an external storage device to store trackinginformation including only a tracklet TL which is the movementtrajectory of the object of which the degree of conviction is equal toor greater than a threshold. Thereby, a detection region-setter 118 ofthe sixth embodiment sets the detection window W_(D) using only atracklet TL of an object having a high degree of conviction indicating atarget object.

According to the sixth embodiment described above, only a tracklet TL ofan object having a high degree of conviction indicating a person isincluded in the tracking information, it is possible to suppress thesetting of the detection window W_(D) with reference to tracklets TL ofobjects other than a target object which is erroneously tracked.

Meanwhile, in any of the above-described embodiments, a description hasbeen given in which the output controller 116 changes a display mode forthe tracking information on the basis of the evaluation result of theevaluation and analyzer 120, but there is no limitation thereto. Forexample, the output controller 116 may cause a user to perform settingfeedback on the detection line LN_(D) by causing a speaker (not shown)to output the evaluation result of the evaluation and analyzer 120 as asound. For example, the output controller 116 may perform sound guidanceon the contents such as “A detection line LN_(D) set by a userintersects a tracklet TL having an accuracy of detection of ∘∘%”, or thecontents such as “An over-detected state was detected by a detectionline LN_(D) set by a user”.

(Hardware Configuration)

The image-processing apparatus 100 of an embodiment described above isrealized by, for example, a hardware configuration as shown in FIG. 28.FIG. 28 is a diagram showing an example of a hardware configuration ofthe image-processing apparatus 100 of an embodiment.

The image-processing apparatus 100 is configured such that an NIC 100-1,a CPU 100-2, a RAM 100-3, a ROM 100-4, a secondary storage device 100-5such as a flash memory or a HDD, and a drive device 100-6 are connectedto each other by an internal bus or an exclusive communication line. Aportable storage medium such as an optical disc is mounted in the drivedevice 100-6. A program stored in the secondary storage device 100-5 ora portable storage medium mounted in the drive device 100-6 is developedin the RAM 100-3 by a DMA controller (not shown) or the like, and isexecuted by the CPU 100-2, whereby the controller 110 is realized. Aprogram referred to by the controller 110 may be downloaded from otherdevices through a communication network NW.

According to at least one embodiment described above, a target object istracked in a moving image, tracking information including a tracklet TLwhich is a movement trajectory of the tracked object is stored in thestorage 130 or the external storage device, a detection line LN_(D) or adetection region R_(D) for detecting passing through of an object is setin a frame within the moving image on the basis of the tracklet TLincluded in the tracking information, a setting mode for the detectionline LN_(D) or the detection region R_(D) is evaluated on the basis ofthis set detection line LN_(D) or detection region R_(D) and thetracklet TL, and the tracking information is output in an output modebased on the evaluation result, whereby it is possible to improve theaccuracy of detection of an entering or leaving person.

The above embodiment can be represented as follows.

An image-processing apparatus, including:

an output device configured to output information;

a storage configured to store a program; and

a processor,

wherein by executing the program, the processor

tracks an object which is a target in one or more time-series images,

stores tracking information including movement trajectories of one ormore of the tracked objects in the storage or an external storage,

sets a detection region for detecting passing through of the object onthe basis of the movement trajectory of the object, on the image, inaccordance with a user's operation,

evaluates a setting mode for the detection region on the basis of thetracking information stored in the storage or the external storage andthe set detection region, and

outputs the tracking information stored in the storage or the externalstorage, from the output unit in an output mode based on an evaluationresult of the setting mode for the detection region.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An image-processing apparatus, comprising: anoutput unit configured to output information; a tracker configured totrack an object which is a target in one or more time-series images; astorage controller configured to store tracking information includingmovement trajectories of one or more of the objects tracked by thetracker, in a predetermined storage; a setter configured to set adetection region for detecting passing through of the object on thebasis of the movement trajectory of the object, on the image, inaccordance with a user's operation; an evaluator configured to evaluatea setting mode for the detection region on the basis of the trackinginformation stored in the predetermined storage and the detection regionwhich is set by the setter; and an output controller configured tooutput the tracking information stored in the predetermined storage,from the output unit in an output mode based on an evaluation result ofthe evaluator, wherein the evaluator evaluates a setting mode for thedetection region, for each movement trajectory selected by the user,according to a presence or absence of intersection between one or moremovement trajectories and the detection region which is set by thesetter, the one or more movement trajectories being selected by the userfrom among movement trajectories of one or more of the objects includedin the tracking information stored in the predetermined storage.
 2. Theimage-processing apparatus according to claim 1, wherein the evaluatorevaluates a high setting mode for the detection region with respect to amovement trajectory that intersects the detection region among the oneor more movement trajectories selected by the user, and evaluates a lowsetting mode for the detection region with respect to a movementtrajectory that does not intersect the detection region.
 3. Theimage-processing apparatus according to claim 2, wherein the output unitincludes a display configured to display an image, and the outputcontroller performs a display for further highlighting the movementtrajectory for which a higher evaluation of the setting mode for thedetection region is made by the evaluator, on the display.
 4. Theimage-processing apparatus according to claim 1, wherein the evaluatorevaluates a low setting mode for the detection region with respect to amovement trajectory that intersects the detection region among the oneor more movement trajectories which are not selected by the user, andevaluates a high setting mode for the detection region with respect to amovement trajectory that does not intersect the detection region.
 5. Theimage-processing apparatus according to claim 4, wherein the output unitincludes a display configured to display an image, and the outputcontroller performs a display for further highlighting the movementtrajectory for which a lower evaluation of the setting mode for thedetection region is made by the evaluator, on the display.
 6. Theimage-processing apparatus according to claim 1, wherein the outputcontroller further outputs the detection region, set on the image by thesetter, from the output unit, in an output mode based on the evaluationresult.
 7. The image-processing apparatus according to claim 6, whereinin a case where one or more movement trajectories selected by the userfrom among movement trajectories of one or more of the objects includedin the tracking information stored in the predetermined storage and thedetection region intersect each other, the evaluator evaluates thesetting mode for the detection region in accordance with an intersectionposition on the one or more movement trajectories.
 8. Theimage-processing apparatus according to claim 7, wherein the evaluatorevaluates a higher setting mode for the detection region as theintersection position approaches a center of the movement trajectory,and evaluates a lower setting mode for the detection region as theintersection position becomes farther away from the center of themovement trajectory.
 9. The image-processing apparatus according toclaim 6, wherein in a case where one or more movement trajectoriesselected by the user from among movement trajectories of one or more ofthe objects included in the tracking information stored in thepredetermined storage and the detection region intersect each other, theevaluator evaluates the setting mode for the detection region inaccordance with an intersection position in the one or more detectionregions.
 10. The image-processing apparatus according to claim 9,wherein the evaluator evaluates a higher setting mode for the detectionregion as the intersection position approaches a center of the detectionregion, and evaluates a lower setting mode for the detection region asthe intersection position becomes farther away from the center of thedetection region.
 11. The image-processing apparatus according to claim6, wherein the evaluator evaluates the setting mode for the detectionregion in accordance with an angle between one or more movementtrajectories selected by the user from among movement trajectories ofone or more of the objects included in the tracking information storedin the predetermined storage and the detection region.
 12. Theimage-processing apparatus according to claim 11, wherein the evaluatorevaluates a higher setting mode for the detection region as the angleapproaches 90 degrees, and evaluates a lower setting mode for thedetection region as the angle approaches 0 degrees or 180 degrees. 13.The image-processing apparatus according to claim 6, wherein theevaluator evaluates the setting mode for the detection region inaccordance with a distance between one or more movement trajectorieswhich are not selected by the user from among movement trajectories ofone or more of the objects included in the tracking information storedin the predetermined storage and the detection region.
 14. Theimage-processing apparatus according to claim 13, wherein the evaluatorevaluates a lower setting mode for the detection region as the distancebecomes shorter.
 15. The image-processing apparatus according to claim6, wherein the evaluator evaluates the setting mode for the detectionregion in accordance with an angle between one or more movementtrajectories which are not selected by the user from among movementtrajectories of one or more of the objects included in the trackinginformation stored in the predetermined storage and the detectionregion.
 16. The image-processing apparatus according to claim 15,wherein the evaluator evaluates a lower setting mode for the detectionregion as the angle approaches 90 degrees.
 17. The image-processingapparatus according to claim 2, wherein the detection region is adetection line.
 18. The image-processing apparatus according to claim 1,wherein the setter further sets candidates for one or more of thedetection regions on the image, and determines a candidate, selected bythe user from among the candidates for one or more of the detectionregions set on the image, as the detection region.
 19. Theimage-processing apparatus according to claim 1, wherein the trackerderives a degree of conviction indicating a degree of probability of anobject to be tracked in the one or more time-series images being anobject which is the target, and the storage controller stores thetracking information, including a movement trajectory of an object ofwhich the degree of conviction derived by the tracker is equal to orgreater than a threshold among one or more of the objects tracked by thetracker, in the predetermined storage.
 20. The image-processingapparatus according to claim 1, wherein the movement trajectory of theobject includes a moving direction of the object, and the outputcontroller outputs the tracking information, stored in the predeterminedstorage, from the output unit, in an output mode according to the movingdirection of the object.
 21. An image-processing system, comprising: theimage-processing apparatus according to claim 1; and one or more camerasconfigured to generate the one or more time-series images.
 22. Animage-processing method comprising causing a computer to: track anobject which is a target in one or more time-series images; storetracking information including movement trajectories of one or more ofthe tracked objects in a predetermined storage; set a detection regionfor detecting passing through of the object on the basis of the movementtrajectory of the object, on the image, in accordance with a user'soperation; evaluate a setting mode for the detection region on the basisof the tracking information stored in the predetermined storage and theset detection region; and output the tracking information, stored in thepredetermined storage, from an output unit configured to outputinformation, in an output mode based on an evaluation result of thesetting mode for the detection region, evaluate a setting mode for thedetection region, for each movement trajectory selected by the user,according to a presence or absence of intersection between one or moremovement trajectories and the detection region, the one or more movementtrajectories being selected by the user from among movement trajectoriesof one or more of the objects included in the tracking informationstored in the predetermined storage.
 23. A non-transitorycomputer-readable storage medium having a program stored therein, theprogram causing a computer to execute: a process of tracking an objectwhich is a target in one or more time-series images; a process ofstoring tracking information including movement trajectories of one ormore of the tracked objects in a predetermined storage; a process ofsetting a detection region for detecting passing through of the objecton the basis of the movement trajectory of the object, on the image, inaccordance with a user's operation; a process of evaluating the settingmode for the detection region on the basis of the tracking informationstored in the predetermined storage and the set detection region; and aprocess of outputting the tracking information, stored in thepredetermined storage, from an output unit configured to outputinformation, in an output mode based on an evaluation result of thesetting mode for the detection region, a process of evaluating a settingmode for the detection region, for each movement trajectory selected bythe user, according to as presence or absence of intersection betweenone or more movement trajectories and the detection region, the one ormore movement trajectories being selected by the user from amongmovement trajectories of one or more of the objects included in thetracking information stored in the predetermined storage.